Papers with k-NN classifier
Cache me if you Can: an Online Cost-aware Teacher-Student framework to Reduce the Calls to Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Prompting Large Language Models (LLMs) performs impressively in zero- and few-shot settings. |
| Approach: | They propose a framework that allows reducing calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side. |
| Outcome: | The proposed framework reduces calls to LLMs by caching previous LLM responses and using them to train a local inexpensive model on the SME side. |